A precise high-dimensional asymptotic theory for boosting and minimum-?1-norm interpolated classifiers

This paper establishes a precise high-dimensional asymptotic theory for boosting on separable data, taking statistical and computational perspectives. We consider setting where the number of features (weak learners) p scales with sample size n, in an overparametrized regime. Under class models, we provide exact analysis generalization error when algorithm interpolates training data maximizes empirical ?1-margin. Further, explicitly pin down relation between test optimal Bayes error, as well proportion active at interpolation (with zero initialization). In turn, these characterizations answer certain questions raised (Neural Comput. 11 (1999) 1493–1517; Ann. Statist. 26 (1998) 1651–1686) surrounding boosting, under assumed generating processes. At heart our lies in-depth study maximum-?1-margin, which can be accurately described by new system nonlinear equations; to analyze this margin, rely Gaussian comparison techniques develop novel uniform deviation argument. Our arguments handle (1) any finite-rank spiked covariance model feature distribution (2) variants corresponding general ?q-geometry, q?[1,2]. As final component, via Lindeberg principle, establish universality result showcasing that scaled ?1-margin (asymptotically) remains same, whether covariates used arise from random or appropriately linearized matching moments.